Skip to content

fix(google): cap max_tokens at 65536 on the Gemini 3.8 Live models - #591

Merged
Kamilbenkirane merged 1 commit into
mainfrom
fix/google-live-max-tokens
Nov 9, 2026
Merged

Kamilbenkirane merged 1 commit into
mainfrom
fix/google-live-max-tokens

Conversation

@Kamilbenkirane

Copy link
Copy Markdown
Member

gemini-3.8-live and gemini-3.8-live-extended-thinking set no max_tokens constraint, so celeste sends any value. Google documents a fixed output limit for both. The model pages (Gemini 3.8 Live, Gemini 3.8 Live Extended Thinking) each list "Input token limit 131,072 / Output token limit 65,536". GET /v1beta/models/{id} returns outputTokenLimit: 65536 for both. The GenerateContent reference defines maxOutputTokens as "The maximum number of tokens to include in a response candidate. Note: The default value varies by model, see the Model.output_token_limit attribute". The Live API reference takes a GenerationConfig in BidiGenerateContentSetup.generationConfig and does not list maxOutputTokens among the unsupported fields. On Google Cloud, the Gemini 3.8 Live page lists "Maximum output tokens 64K", the same 65,536. This is a documented output figure, which is what CLAUDE.md's MAX_TOKENS rule requires.

The constraint has an effect because max_tokens already reaches the setup frame. MaxTokensMapper (GOOGLE_PARAMETER_MAPPERS, wire MaxOutputTokensMapper) writes generationConfig.maxOutputTokens, and GoogleLiveClient._build_request puts the result under setup. The existing wire-contract row covers this mapping.

Fix:

  • src/celeste/modalities/live/providers/google/models.py: LiveParameter.MAX_TOKENS: Range(min=1, max=65536) on both entries. This is the same shape as the Google text models.
  • No test change: the mapper list is unchanged, and test_parameter_wire_contracts.py already has the Live maxOutputTokens row.

Validation:

  • make ci passes (762 unit tests).

  • Live, Gemini API key. One turn per case through celeste.live.connect(...) on wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent, prompt "Say hello in one short sentence.":

    • Both models, max_tokens=65536: the setup frame carries generationConfig.maxOutputTokens: 65536, setupComplete arrives, and the turn ends with turnComplete (audio plus transcript).
    • Both models, max_tokens=65537: rejected locally with ConstraintViolationError: Must be between 1 and 65536, got 65537.
  • The field is honored on gemini-3.8-live, with the prompt "Tell me a long, detailed story about a lighthouse keeper":

    max_tokens response tokens thinking tokens
    150 71 57
    400 275 48
    not set 1,486 78

API reference vs. live serving (recorded per CLAUDE.md): the Live setup only checks that the value is positive.

  • maxOutputTokens: 0, sent through extra_body to skip celeste's check, closes the socket with 1007 "max_output_tokens must be positive". That confirms min=1.
  • 65537 and 1000000 sent the same way are accepted, and the turn completes normally.
  • So nothing live enforces max=65536. The cap comes from the documented output-token limit and outputTokenLimit, as it does for the Google text models.
  • Side observation, not changed here: with max_tokens=20 on gemini-3.8-live, the server sends setupComplete and then no modelTurn or turnComplete for 45 s. The small budget seems to go entirely to thinking tokens.
  • Google Cloud (GoogleADC) was not called live: the local ADC project lacks aiplatform.endpoints.predict.

@Kamilbenkirane
Kamilbenkirane merged commit 7e93e92 into main Nov 9, 2026
10 checks passed
Kamilbenkirane added a commit that referenced this pull request Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant